Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added reframer_ts_vtt filter. #7

Merged
merged 9 commits into from
Jan 23, 2025
Merged

added reframer_ts_vtt filter. #7

merged 9 commits into from
Jan 23, 2025

Conversation

jadarve
Copy link

@jadarve jadarve commented Dec 23, 2024 •
edited
Loading

Given a Transport Stream file containing Metadata as PES packets with metadata format as "WVTT", the new rftsvtt filter reads the content of each PES packet and creates a WebVTT cue output that can be used by the dasher to create a subtitles track for both DASH and HLS.

image

Running GPAC as:

/home/juan/workspace/git/gpac/bin/gcc/gpac \
  -i /home/juan/workspace/notebooks/GStreamer/out_subtitle.ts \
  rftsvtt \
  -o "/home/juan/workspace/notebooks/GStreamer/gpac/hls/playlist.m3u8:gpac:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true" \
  -graph

creates a filter graph as follows:

Filters connected:
fin (src=/home/juan/workspace/notebooks/GStreamer/out_subtitle.ts) (idx=1)
-(PID out_subtitle.ts) m2tsdmx (dyn_idx=4)
--(PID P1T2) rftsvtt (idx=2)
---(PID P1T2) dasher (dyn_idx=7)
----(PID manifest_m3u8) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/playlist.m3u8:gpac:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true) (idx=3)
----(PID P1T2)          ufvtt (dyn_idx=11)
-----(PID P1T2) writegen (dyn_idx=12)
------(PID P1T2) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/segment_track65_init.vtt:gfopt:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true:noinitraw:frag:xps_inband=no:psshs=moov) (idx=8)
----(PID P1V3)          mp4mx (dyn_idx=13)
-----(PID P1V3) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/segment_track66_init.mp4:gfopt:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true:frag:xps_inband=no:psshs=moov:mime=video/mp4) (idx=9)
----(PID P1A4)          mp4mx (dyn_idx=14)
-----(PID P1A4) fout (dst=/home/juan/workspace/notebooks/GStreamer/gpac/hls/segment_track67_init.mp4:gfopt:buf=-150:seg_sync=no:segdur=1.0:cdur=1.0:hlsc=true:cmaf=cmf2:dmode=static:template=segment:rawsub=true:frag:xps_inband=no:psshs=moov:mime=audio/mp4) (idx=10)
--(PID P1V3) rfnalu (dyn_idx=5)
---(PID P1V3) dasher (dyn_idx=7)
--(PID P1A4) rfadts (dyn_idx=6)
---(PID P1A4) dasher (dyn_idx=7)

Notice that, as this is HLS, the rawsub=true flag is used. The output playlist defines the subtitles track:

#EXTM3U
#EXT-X-VERSION:6
#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="1",AUTOSELECT=YES,URI="playlist_1.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=2117639,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION=640x480,FRAME-RATE=25,AUDIO="audio",SUBTITLES="subs",CLOSED-CAPTIONS=NONE
playlist_2.m3u8

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="3",AUTOSELECT=YES,URI="playlist_3.m3u8",CHANNELS="2"

and playlist_1.m3u8

#EXTM3U
#EXT-X-TARGETDURATION:1
#EXT-X-VERSION:6
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-INDEPENDENT-SEGMENTS
#EXTINF:1,
segment_track65_1.vtt
#EXTINF:1,
segment_track65_2.vtt
#EXTINF:1,
segment_track65_3.vtt
#EXTINF:1,
segment_track65_4.vtt
#EXTINF:1,
segment_track65_5.vtt
#EXTINF:1,
segment_track65_6.vtt
#EXTINF:1,
segment_track65_7.vtt
#EXTINF:0.8,
segment_track65_8.vtt

#EXT-X-ENDLIST

where each .vtt file contains 1 second of data.

estigma88
estigma88 reviewed Jan 7, 2025
View reviewed changes
src/filter_core/filter_register.c Show resolved Hide resolved
src/filters/dmx_m2ts.c Outdated Show resolved Hide resolved
src/filters/dmx_m2ts.c Outdated Show resolved Hide resolved
src/filters/dmx_m2ts.c Outdated Show resolved Hide resolved
src/filters/reframe_ts_wvtt.c Show resolved Hide resolved
@jadarve jadarve merged commit 5f75468 into integration Jan 23, 2025
@jadarve jadarve deleted the VID-7982_support_live_subtitles branch January 23, 2025 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@estigma88 estigma88 estigma88 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jadarve @estigma88